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RADIAL FRACTIONAL LAPLACE OPERATORS 
AND HESSIAN INEQUALITIES 

FAUSTO FERRARI AND IGOR E. VERBITSKY 



Abstract. In this paper we deduce a formula for the fractional Laplace operator (— A) s on 
radially symmetric functions useful for some applications. We give a criterion of subharmonicity 
associated with (— A) s , and apply it to a problem related to the Hessian inequality of Sobolev 
i type: 



r \ k ik+i r 

/ (-A)wJ dx<C -uF k [u}dx, 



where Fk is the fc-Hessian operator on R", 1 < k < 2, under some restrictions on a fc-convex 
function u. In particular, we show that the class of u for which the above inequality was 
established in |FFV| contains the extremal functions for the Hessian Sobolev inequality of X.-J. 
Wang |W1| . This is proved using logarithmic convexity of the Gaussian ratio of hypergeometric 
functions which might be of independent interest. 

Ph 

< 

1. Introduction 
Let n > 2. For every s £ (0, 1) and a locally integrable function u : M n — > K such that 

(1-1) / ; ' '\ + 2s dx<+oo, 

> . JR n (1+ I x I 2 ) - 2~ 

T-i- ■ we define the s-fractional Laplace operator as follows: 

(1.2) (-Aru(x) = c s , n [ ^ X) ~^l dy, 

^ ; 7r" \x-y \ n+IS 

where c s ^ n is a positive normalization constant. The integral is convergent (in the principal 
value sense if ^ < s < 1) if u is, for instance, a bounded C 2 function. For any u satisfying (jl.ip . 
(— A) s u is defined in the sense of distributions: 

3: (1-3) <(-A) s u, h)= I u(-A) s hdx, 

H ''■" 

for all test functions h £ C^°(W n ). This is a linear nonlocal operator, that is, roughly speaking, 
if u satisfies (— A) s u(x) = in a domain Q C M. n , then the value of u at any point of f2 depends 
not only on the neighborhood of the point itself but also on the behavior of the function in the 
entire space. We refer to |Lan| and [CaS| for further details. 

Some new relationships with a class of local nonlinear operators, the fc-th Hessian operators, 
have been pointed out in [FFVJ. We recall that the fe-th Hessian operator F^ can be defined in 
several equivalent ways (see e.g. [W2j ) . For a C 2 function u in a domain Q, C R n , we define 
Fk[u], k 6 N, 1 < k < n, as the fe-th symmetric elementary function of the eigenvalues of D 2 u(x), 
the Hessian matrix of u. Just to give an idea of this family of operators we note that F\ [u] = Au 
when k = 1, and F n [u] = det(L> 2 u) when k = n. A function u G C 2 (il) is called fc-convex if 
Fj[u] > for all j = 1,2, ... ,k. This definition was extended to general upper semicontinuous 
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2 F. FERRARI AND I. E. VERBITSKY 

functions by Trudinger and Wang |TWlj . |TW2j . In particular, u is 1-convex if and only if u is 
subharmonic, while u is n-convex if it is convex in the usual sense. 

Clearly, F k are local operators, since it is possible to calculate Fk[u] pointwise by using 
second order partial derivatives of u. Moreover, fc-th Hessian operators are fully nonlinear while 
s-fractional Laplace operators are linear. Nevertheless, these operators are closely related. It 
was shown in |FFVj (Theorem 2.1) that, under certain assumptions on u discussed below, the 
following inequality holds (see Proposition 15.81 in this paper for the detailed statement): 

(1.4) / \(-A) s u\ k+1 dx < C Kn f -uF k [u)dx, 

JR n JR" 

where s = rrr, 1 < fe < S, and C k)Tl is positive constant depending only on k and n. The 
converse inequality holds as well with a different constant, for all fc-convex C 2 functions u 
vanishing at infinity (|FFV|, Theorem 3.1). 

In this paper we want to improve our knowledge of the s-fractional Laplace operator by 
studying s-subharmonic functions that are analogous to subharmonic functions for the Laplace 
operator, and in particular the class of fe-convex functions introduced in [FFV] for which (|1.4p 
holds. In fact we will prove below that this class contains extremal functions of the Hessian 
Sobolev inequality due to X.-J. Wang [Wl], |W2] : 



(1.5) / \u\ q dx) <Ci n / -uF k [u]dx 



i i 

q ( I" \ fc+1 



/ fc,n 

where 1 < k < S, q = „_ 2 fc > an d u is a /c-convex C 2 function on W 1 vanishing at oo. We 
remark that (jl.4p implies (jl.5p by the classical Sobolev embedding theorem. 

Our approach is to study the s-fractional Laplace operator and the corresponding notion of 
s-subharmonicity for radially symmetric functions u(x) = u(r), r = \x\. We will prove the 
following result. 

Theorem 1.1. Let s £ (0, 1). For every radial C 2 function u such that 

f' + °° I ii(r\ I 

(1.6) / . ' \ ; ' r n - l dr<+oo, 

V > J q (1+r) n+2, 

the following formula holds: 
(1.7) 

(-A) s n(r) = c s , n r- 2s f " (u{r) - u(rr) + {u{r) - u (-)) T - n+2s ^ t(t 2 - l)- 1 - 2 *£f(r)dr, 
where r = \x\ > 0, x £ W 1 , and 



n — 3 



" ^n-2 ( Vr 2 - sin 2 9 + cos r +*» M ^, „ _ ^ 



H(T) = 27ra n l sin""^ ^ — = ' d9, r > 1, a n -, wi . | 



/o Vr 2 -sm 2 e r(V) 

Clearly, H(t) is a positive continuous function on [l,+oo), with H(t) — t 2s as r — > +oo. 
One can express H(t) in terms of the Gaussian hypergeometric function: 

n 

r(r 2 - 1)- 1 -»H{t) = ^- ) r- 1 - 2s 2 F 1 (a,b,c,r- 2 ) a (r - 1)"^, r > 1, 

where 

n + 2s . n 

a = , b = l + s, c = — . 

2 ' '2 

Note that 2-^1(0, b,c, t~ 2 ) has a singularity of order (r — l) -1 ~ 2s at r = 1, while H(t) is 

continuous at r = 1. The integral in (jl.7p is convergent for any radial C 2 function u satisfying 

OD. 
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For every s £ (0, 1] and u satisfying (jl.ip . we say that u is s-subharmonic in 0, if u : f2 — > 
[— oo, +oo) is upper semicontinuous in fi, and 

-(-A) s u(x) > in D'(n); 

in other words, — (— A) s u is a positive Borel measure in Q. 

Theorem 1.2. Let s G (0, 1]. Let u(r) 6e a radial upper semicontinuous function in M+ suc/i 
that il.6\) holds. If, for all r > and r > 1, 

(1.8) u(r) - u(rr) + (u(r) - u(-))r~ n+2s < 0, 

r 

t/ien -u is s-subharmonic in M. n \ {0}. 

If in Theorem II .21 one assumes additionally that u is bounded above in a neighborhood of 0, or 
more generally limsup r _ >0 ' u ( r ) r n ~ 2s < 0, then u has a removable singularity at (see [BH], p. 
379, Corollary 10.2), i.e., u is s-subharmonic in the entire space M n (with u(0) = limsup r _ !h o u{r)). 

We observe that condition (jl.8|) is of interest for any real s. The following theorem provides 
a more convenient pointwise characterization of (jl.8p . Combined with Theorem 11.21 it gives a 
useful sufficient condition for a radially symmetric function to be s-subharmonic when < s < 1. 

Theorem 1.3. Let s € M.. Let u € C 2 (1R-|-) be a radial function. Then i ti.ffj) holds if and only 
if, for all r > 0, 

u { r ) 

(1.9) u"{r) + (n-2s + l)— — > 0. 



Condition (I1.9P is not necessary for a radially symmetric function u to be s-subharmonic when 
< s < 1 (see examples below). However, the class of functions obeying (|1.9j) is quite rich, and 
contains many interesting s-subharmonic functions. 

In the special case s = - 2fc +1, condition (II. 9|) coincides with the following characterization 
of fc-convex radially symmetric C 2 functions (see |W1| , [W2J ) : 

(1.10) u"(r) + VLZl *M > o, 

k r 

where A; = 1, 2, . . . ,n. 

For /3 > 0, let us consider 

/ /3 (x) = -(l+|x| 2 )-2, xGM n . 

It is easy to see (cf. Corollary 14.11 below) that for any s 6 1, /^ satisfies (|1.8p . and consequently 
(jl.9p . if and only if /3 < n — 2s. Moreover, if < s < 1 and < /3 < n, then fp is s-subharmonic 
in M. n if and only if j3 < n — 2s. 

Now let f3 = f — 2 > 0, where A; = 1, 2, . . . , [|] . Then by (ll.lOj) . //3 is fc-convex, and is known 
to be an extremal function for the Hessian Sobolev inequality (jl.5|) ; see [Wlj . |W2j . (A new 
proof of this inequality, but without the sharp constant, is available; see [Y])- O ur m ai n goal is 
to show that u = f» satisfies the following condition introduced in [FFVJ: 

(1.11) (-A) s [-(-A) s u] k > inM n , 

where s = jttj , 1 < k < ^. Under this condition the enhanced Hessian Sobolev inequality (jl.4p 
was proved in [FFVj (Theorem 2.1). The following theorem demonstrates that, in particular, 
this class of functions is nontrivial. 

Theorem 1.4. Let s = jctt an d ft = § — 2, where 1 < k < ^. T/ien /^ satisfies condition 

EZZJJ. 
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The proof of Theorem 11.41 for k > 2 relies on Theorems 11.21 and 11.31 along with a new loga- 
rithmic convexity property of the Gaussian quotient of hyper geometric functions F 2 , 1 ^"' 1 ' c ^fj — r. 

It is worth observing that when k = 1, the function —u = (— A)2/g with j3 = n — 2 (n > 3), 
fails to satisfy condition (|1.9p with s = ^; however, — u is still ^-subharmonic. The latter is 
easily checked using the superposition property of fractional Laplacians: 

(_A)*[-(-A)*//j] = A/0>O. 

The paper is organized as follows. The proofs of Theorems 11.11 and 11.21 are given in Section 
[2J In Section [3] we prove Theorem 1.3 and discuss the s-subharmonicity property for radially 
symmetric functions. Some examples are given in Section |U In Section [5] we discuss some 
further applications of our results. In particular, we prove a Liouville theorem, the maximum 
principle for radial s-subharmonic functions, and a derivative formula involving the fractional 
Laplacian. In Section [6j we prove Theorem 11.41 using a convexity property of a certain ratio of 
hyper geometric functions. Finally, in Section[71 we prove logarithmic convexity of F 2 / fc°V c +{ — y 

and convexity of F ( 2 a+ \ ^1 j x \ in (— oo, 1) (Theorem 17.11 and Corollary 17. 4p under certain 
restrictions on the parameters a, 6, c using a method developed recently in |KS1] (see also [KS2]). 
The proofs generalize to ratios of generalized hypergeometric functions q +\F q for q > 2. 

2. Proofs of Theorems 11.11 and 11.21 

Proof of Theorem li.il For the sake of convenience, we will drop the normalization constant c S) n 
in (|1.2p when it does not lead to a confusion. Let u be a radial C 2 function in R™ satisfying 
(|l,6p . If u(x) = u(\x\), x = rx', y = py' , and | x' | = | y' \= 1, then 

\»..f~\- f +C ° I f U{r)- U{p) ^n-ljun-l, N 

.,/ |n+2s' 



(-A)- B( x) = / i y |r ;: CT ,:::^ "-^"-'fa') i * 



-oo 

(u{r)-u{p))p n - 1 



+ ™ u(r)-u(p) x 

j,n+2s " 



"- l ~ 2s / (u(r) - u{tt))t 
Jo 



+°° u{r)-u{rT) rn _ x 

ii r 2s 



Notice that 



/ 

J\v' 




\V\ 

is independent of x' G {| y |= 1}. Indeed, suppose that z' & {| y |= 1}. Then there exists a 
unitary matrix Q such that z' = Qx'. Thus, performing a change of variables such that y' = Qw' , 
we get: 



M W '|=1 | x' — TW' \ n+2 

because | detQ |= 1 and | Qv\ — Qv-i |=| v\ — v% I for every ui,U2 G 1 



1 dH n ~\w'), 
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Moreover, 

/ " ) -^dH n - l (y>)= [ - —L dH n -\y') 

J\y'\=i (z' - ry', z' - ry')~i~ -Vl=i (1 - 2r(y', z') + r 2 )^ - 



sin™ ;> ■ 



2vra n / — — . rffl, 

/o (l-2rcos6l + r 2 )^- 



where 

n-3 r-w 5=2 

7T 2 



i,—i ■'0 



fc=l" yu ^2 ) 

Notice also that 

-2 _ n \2 



1 - 2r cos + t z > 1 - 2t + t a = (1 - t) z 
Thus, for every r^l, 

-d0 



'o (1 - 2r cos 6> + t 
is bounded, while for r = 1, 



n + 2s 
^ I 9 





(2- 


. n + 2s 

-2cos6»)^- 


We denote 






(2.1) 




K(r) = 2Tra n 



* m " 2 ' } an-2 9 -n~2s^ e -2-2s^ &g g ^ Q 



sin™" 2 

n ,n+2s QP) 

o (l-2rcos6l + r 2 )^ 

and study the behavior of the integral with respect to r. 
Let t > 1. We perform a change of variable as follows: 

sin sin i/j 

\/r 2 - 2r cos 9 + 1 r 

Consequently, 

sin 2 t/» ± cos V^v t 2 — sin 2 -0 
cos = . 

T 

Hence 

a d Q -1 ■ , I o ; /~1 . 2 , COS 2 ')/' 

sin #-7-7 = t sin ?/> 2 cos ^ -f V r sm V' -F 



cos ip + y t 2 — sin V' 



(99) d ^ V V v^-sin 2 ^ 

sin^ 
ryr 2 — sin 2 ?/> 
Moreover 

r sin = sin ip y r 2 — 2t cos 9 + 1 



/ 






smifiy t 2 - 


- 2(sin 2 ip ± cos ipy t2 ' 


- sin 2 t/0 + 1 


j 






smi/j\ t 2 - 


- sin 2 V> =F 2 cos ^ y r 2 - 


- sin ^ + 1 — sin tp 



sin ip \ t 2 — sin ^ =p 2cosip\/ t 2 — sin ^ + cos 2 i/> 



(2.3) 



smip\l (a/t 2 — sin t/» =p cos^) 2 = sin ip \ \J t 2 — sin tp + cosip 
We obtain, by plugging the relation (|2.3p into (|2.2p : 
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dO 



/„ .s dip | y/7-2 _ g ^ n 2 ^ -p cog ^ I y' T 2 _ g j n 2 ^ 

1 - cos tb 

yr- sin ^y 

Hence, 



cos ib^f \/t 2 — sin ^ 



sin™ 2 1 



/■7T 

jo (1 



d0 



(1 - 2r cos 6> + t 2 )- 

( — - ) + sm 6 1 T . = = d^ 

r \ yr - sin z ibj 

= r 2 +2s r ( sinv; )n+2s /. n ^ ! r - _ sin2 - ^ cos ^ a - ' L cos ^ \ d ^ 

/ 2>5 ) -A) r V ! / V V^ 2 -sin 2 V/ 

„2-n / _±__n-2 _/. /1 /To ~2~T i\\ li_ COS f/' 



r z n / sin™ ^ ^ I I V T sin V> =F cos tb | 1 q= = = d^ 

Jo V J \ \Jr 2 -sin 2 ^ y 

t 2 -™ / ^n"- 2 ,/, -===dl\) 

( a/ r 2 — sin 2 ^ =p cos ib) 1+2s yr 2 — sin 2 ib 







r 2 ~> 2 - 1)-^- / ^-2jVr 2 -sm 2 ^ + cos^+ 2 ^ 



'0 a/t 2 — sin 2 ib 

Let us denote 



/ n „,, r r, 2 , (\/^ 2 -sin 2 V + cos^) 1+2s ,, 

(2.6) H(T) = 2ira n sin™" 2 ^— 7== ~ */>• 

Jo a/ r 2 — sin 2 ^y 

Then -ff(r) = r n_2 (r 2 — l) 1+2s K(t). Clearly, H(t) is a positive continuous function on [1, +00) 
such that H(t) ~ t 2s as r — > +00. Moreover, it follows from (|2.ip that, as was mentioned in 
the Introduction, .ff(T) can be expressed in terms of the hypergeometric function 2-F1 (a, b, c, r) 
(see [MPS] , p. 55): 

11 

tf (r) = t~ 2 - 2 V - l) 1+2s |Jy 2*1 (a, b, c,r'% r> 1, 



where 



On the other hand 



n + 2s n 

a = — - — , = 1 + s, c = — . 



(2.7) 



» - 1 u ( r ) — u(rr) _ 1 __. . , 

(-A) s u(r) = J 2s ' T n - l K{T)dT 



r 



1 u{r) - 2 f rT K^K(r)dr + f°° < T )-< rT \ {T 2 _ l)-^H(r)dr. 



/o r 

Let us consider the following integral 

- 1 u(r) -«("") _n-l 

/o 



/' U{r) - U{rT K^K(r)dr, 
Jo r 
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and perform the following change of variable: £ = ~. Then 

(2.8) Jo r \ ,; , n e 

r+oo u (r) - u(j) . i 

Notice that 

K(l) = 2,a n e +2s [" - - Sln "' 2g . rW 
£ Jo (t 2 -2£cos6 + iy 



2 

Hence, 

(2.9) 



2 r^K(r)dr = S r^f^R 



+ °° n(r) J^ r-r+^R = / + °° " (r) 2 / (|) r 1+2 ^ 2 -^ 2 - i)- 1 - 2 -^)* 

+oo u ( r ) -u(|) 1 



/ " MC %^V -^ 2 - ir^m- 



Thus 



- -v / r 



(2.10) (-A)MO = r~ 2 * y U(r) - u(r£) + («(r) - u(-))£- n+2s J £(£ 2 - l)" 1 " 2 ^ (£R- 
The convergence of the integral in (|2.10p is discussed below in Sec. [3J D 

We now deduce Theorem 1 1.2 1 from Theorem [1J] using mollification defined by means of Mellin's 
convolution which preserves condition (|1.8p . 



Proof of Theorem \1.2l Suppose u is a radially symmetric upper semicontinuous function satis- 
fying the conditions 

r + °° I ii(r\ I 

(2.11) / ' Y' r"- 1 dr<+oo, 

and 

(2.12) u{r) - u(r£) + (u(r) - u(j))C n+2s < 0, for all r > 0, £ > 1. 

Let us show that u is s-subharmonic, i.e., — (— A) s n > in the sense of distributions. Let 
(j) £ Cq°(R) so that <j) > 0, <j> is supported in the interval |r| < ro, where < vq < +oo, and 
f K 4>(y)dy = 1. For e > 0, define the approximate identity on R + by: 

(2.13) e (r) = ~<£ (^f 1 ), ^>0- 
Then clearly, for every e > 0, 

/•+00 J 

(2.14) / e (r)- = 1. 

We observe that (|2.12[) is invariant under the Mellin convolution: 

(2.15) u e (r) = <f> e (-J u(t)j, t > 0. 
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Indeed, integrating both sides of (|2.12p against </> e (^)*, we obtain: 
(2.16) 



u e (r) - u e (rO + (u e (r) - u e (-))C n+2s < 0, for all r > 0, f > 1. 



Moreover, obviously u e G C°°(]R+), and 
/* + °° |u e (r)|r n - ] 
io 



(2.17) 



+°° \ v (r)\ r n ~ x 
dr<C I , f\ 9 dr < +oo, 





(1 + r) n+2s ~ 7o (1 + r) n+2s 
where C = C(e,ro,s,n) is a positive constant. Indeed, by Fubini's theorem, 

+ °°, /m /' +oc A n £" ,/^dXdt „ f +co \u(t)\t n dt 

/n K*)l / tttuot^Wt - < C / W 



.70 







(1 + Xt) n + 2 



+ t) n + 2s t 



< +oo, 



<^0 „er ^ 



and (l2~T4jl 



where the last estimate follows since </> e (A) vanishes outside the interval (e~ 
holds. 

Let h £ Cg°(R n \ {0}) be a nonnegative test function supported in < \x\ < R < +oo. We 
observe that, since h is compactly supported, it follows from (11.2p that (— A) s h £ C°°(]R n ), and 

(2.18) \(-A) s h(x)\< ( 



(i + W 



,n+2s ' 



x 6 



In particular, 



((-A) s u, h) 



I u(-A) s hdx 



is well-defined in terms of distributions (see |Lan] . Sec. 1.6). By Theorem 11.11 and (12. 16|) . 

(-A) s u e > 0, and hence, using (I2.11J) and recalling that u e £ C°°(R n \ {0}), we obtain: 



(2.19) 



((-A) 8 u e , h) = (u e , (-A) s h) 



Ur 



-A) s hdx> 0, 



for every h £ C^(R n \ {0}), h > 0. 

It remains to prove the approximation property: 

(2.20) lim / u e (-A) s hdx= I u(-A) s hdx. 

Denote by ip(t) the spherical mean of (— A) s h: 

ip{t)= i (-AYhitx^dH 11 - 1 ^'), t>0. 

J\x'\=l 



Then 



We estimate: 



r +,x dt 

[u(x) - u e (x)] (-A) s h(x) dx = / [u(t) - u e (t)} i/>(t) t n — . 

Jo t 



°°[u(t)-u e (t)]mt n T 

o l 



+oo 



U (t) 



< 




■oo 



^{t)t n 



+oo 



t' r 



dt 

T 



|«(t)| / \t n m-r n ip(r)\cf> e {~)-^. 
Jo t r t 



We recall that, since (j)(y) is supported in the interval \y\ < tq, it follows that (pe(j) is supported 
in the interval where | log j \< er^. Hence, in the above estimates we can assume: 



~ t 
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From this we deduce: 

(2.21) \r-t\<5t, 

where S = e er ° — 1 — > as e — > 0. 

We notice that, since (— A) s h £ C°°(IR n ), it follows that its spherical mean ip is infinitely 
differentiable on [0, +oo) (see, e.g., [Lan| . Sec. 1.6). We will need the following estimates: 

( 2 - 22 ) M*)l < i ^r^r, W$)\ < t r^r^r, t > 0, 

v ' \~r\l\— M\ 4-\n+2s ' ' r v ' ' — l\ _i_ f\n+2s+l ' — ' 

where C depends on h. Indeed, h{x) = for |x| > R, and consequently (jl.2p yields, for t > R: 
m = -c s , n [ h(y) f J—^dH n -\x') dy, 

J\y\<R J\x>\=l\y-tx\ n+2S 

V/(£) = c s , n (n + 2s) [ h(y) f - ~ ^ dfT 1 ^) dy 

J\y\<R J\x'\=l \V — tx I 

= c s , n (n + 2.) t / My) / J, w+2s+2 ^ n ~V) <fo- 

Here |y - ix'| > t - R > § for i > 2.R, which yields (J2.22J) for i > 2R. Since V is infinitely 
differentiable on [0, +oo), it follows that (J2T22"]) holds for all t > 0. 
Let Vt(£) = t n ip(t). Clearly, estimates (^22|) yield: 

fi j.n s-t ±n—\ 

(2-23) hM*)l< (1 + t)w+2a , N4(*)l< (1 + t)w+2 „ ^>0. 

Invoking the mean value inequality we estimate: 

A n-i 



|f>(£) - r n 4>{r)\ = |Vi(t) - ^i(r)| < |r - t| |V4(A)| < C \r - t\ 



( 1 + A )n+2 S ' 

for some A between t and r. Assuming by (|2.21|) that \r — t\ < 5t, we see that i(l — 5) < A < 
t(l + 5). Combining the preceding estimates, we obtain: 

\em - r^(r)| < ^^fll 1 (TT^- 
Using this together with ()2.14p . we conclude: 

where the right-hand side is finite by (I2.11J) . Letting e — >• 0, and hence 5 — > 0, we conclude the 
proof of the approximation property (|2.2(jp . □ 

3. Fractional subharmonicity for radially symmetric functions 
In this section we study further the condition 

(3.1) u(r) - u(ri) + (u(r) - u(j))C n+2s < 0, for all r > 0, f > 1, 

which ensures that a radially symmetric C 2 function u is s-subharmonic, i.e., — (— A) s u > 0, 
for < s < 1. However, this condition makes sense for any real s. We will show that for C 2 
functions u it is equivalent to: 

(3.2) u"(r) + (n + 1 - 2s)— ^ > 0, for all r > 0, 

r 

which is stated as Theorem 11.31 in the Introduction. 
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Suppose u is a radial C 2 function. It is worth noting that in a neighborhood of 1, whenever 

r (P — -\\2(pn+l-2s _ i\ 7 /"M 1 

«(r) - u(rfl + (u(r) - u&jr" 4 *' = -^(r)r ^ -^|- - lj - r 2 ^i% - 1) 2 (1 + — 



£" s \ / £ n+1— 2s (£ — 1) 2 £n+2-2s- 

r— ((e - 1) 2 ) = - y*- 1)2 ^T + 1) «'(r) + ^fP^^-'W) + «t - 1) 2 
_1 2 (e-i) 2 (r +2 - 2s + i) , 

9 tn+2— 2s ' 



where 



„ , 2£(r l +l-2 S + ^(£-l) + ^±p^(£-l) 2 + (£-l) 



*« = u " + — 2 (i + e + 2 - 2 -) — ^^7 + o(r ' (e " 1} 



2g(n + l-2 S ) u' 

l + gn+2-2s +g(g~ 1 ))7 



it" + (n + 1 - 2s) 



«' 



r 



(n + l-2s)(2^-l-r +2 ~ 2s ) + ^K - 1) + ^=^K - 1) 2 t + o(£ - l) 2 ) 



(1 _|_ £n+2-2s\ 



U 



+ o(r, (C - l) 2 ) = u" + (n + 1 - 2s)(l + h{£ - 1))^, 

where lim^_ ;hl + ^Ji = 1- 

In particular, the integral in (jl.7p is convergent in a neighborhood of 1, since for any fixed 
r > it behaves as (1 — £) 1_2s that converges whenever < s < 1. Moreover, by (|2.1ip it 
converges when £ — > +oo. Hence the integral in (jl.7p is convergent for every C 2 function u 
satisfying (|2.1ip . 

The above calculation demonstrates that condition (|3.ip is closely related to condition (|3.2p 
for any s£l, which is proved below. 

Proof of Theorem \l.S\ Let s £ R. Suppose r > and £ > 1. Dividing both sides of (13. ip by 
(£ — l) 2 and passing to the limit as £ — )■ 1 + , we deduce (|3.2p . 

Conversely, suppose that (I3.2p holds, and u'(r) > 0. Let 7 = n — 2s. Making a substitution 
r = e*, £ = e^, where t£l, and x > 0, we rewrite f)3. If) in the equivalent form: 

(3.3) (v(t + x) - u(t))e 7:c + u(< - x) - v(t) > 0, for all * 6 R, x > 0, 
where v(t) = u(e l ), while (J3.2J) is equivalent to 

(3.4) v"{t) + jv'(t) > 0, for all t G R. 

We next fix t £ R, and let </>(x) = u(x + t) — f (t), where x G R. Then clearly (f)(0) = 0, 
0'(O) = v'(t), and (I3.4p is equivalent to 

(3.5) 4>"(x) + 70' (a?) > 0, for all iER. 
We need to prove: 

(3.6) (j)(x)e <x + <p{-x) > 0, x>0, 

which is equivalent to (|3.3p . It is not difficult to deduce fj3.6[) from f|3.5j) using Gronwall's 
inequality (see [Har| . Sec. 3.1, Theorem 1.1) in the case 7 > 0. 

Let us prove this directly for all 7 G R. By (|3.5p . it folows that <j)'(x)+jcj)(x) is non-decreasing 
on R. Since <f>(0) = 0, we obtain: 

(3.7) <f>'(x) + j(j)(x) > <f>'(0), for all x > 0. 
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Let 

i/j(x) = 4>(-x) + 4>(x)e rx , x <E M. 
We next show that ip'(x) > for all x > 0. By $£7$ , 

if/(x) = -tf>\-x) + (<f>'(x) + J<j>(x)) e 7X > -<f/(-x) + 0'(O)e 7X , for all x > 0. 
It remains to prove the inequality 

(3.8) 4>'{-x) < 0'(O) e 7X , for all x > 0. 

Let g(x) = e~ lx (j)'(—x). Then 

g'{x) = _ 7e -7^'(_ x ) _ e -YX(j,»(- x ) = _ e -7^(0"(_ a .) +7 ^(_ s )) < 0, 

by (|3.5p . Hence g is nonincreasing, and consequently g(x) < g(0) for all x > 0. This proves 
(|3.8p . Thus i/>'(a;) > for all x > 0, and since ^(0) = 0, we deduce ip(x) > for all x > 0. This 
proves (|3.6p . which in its turn yields (|3,ip . 

D 

4. Examples of s— subharmonic functions and further remarks 
We consider the fundamental solution of the /c-Hessian operator F^, 

u(x) = - | x |" ( ^" 2) , iel n , 

for 1 < k < 5, which satisfies the equation 

in the viscosity sense (see |TW2j . [W2| ) . 

It is not difficult to verify directly that wis a radial function satisfying the condition 

(4.1) u(r) - u(r£) + (n(r) - u(^))£~ n+2s < 0, for all r > 0, f > 1, 

if s = " 2fc + 1, or equivalently n — 2s = ? — 2. 

More generally, for /J > 0, let us consider the function 

u(x) = - | x | _/3 , x G M n . 
If /3 = n — 2s, we have: 

u ( r \ — u(r£) -r _ ( n_2s ) + r -(n-2«)^- (n-2«) 
u(§) — «(r) _ r -(n-2s)£n-2s _|_ r -(n-2s) 



^ 



1 i t— (n— 2s) 

l±l__ =r (n-2 S )^ 



_fn—2s _i_ J 

It follows that, for /3 = n—2s and s < ■£, condition (|4.ip turns into an equality for all r > 0, £ > 1. 
Unfortunately the A;-energy of u(x) = — | x |~^fc _ > is unbounded, and so such functions cannot 
be used in Hessian inequalities of the type (jl.4p or (jl.5p discussed above. 
This suggests considering the function 



2\-4 „ r- TO" 



/ /3 (x) = -(l+|xr)-2, x€ 

for /3 > 0. When /3 = P — 2, / is a /c-convex function of finite A:-energy. Moreover, /^ is known 
to be an extremal function for the Hessian Sobolev inequality f)1.5|) (see [Wl], [W2| ) . 

Corollary 4.1. Let s£R and j3 > 0. T/ien /« satisfies il.8\) . and consequently il.9\) . if and 
only if (3 < n — 2s. Moreover, if < s < 1 and < /3 < n, £/zen /^ zs s-subharmonic in W 1 if 
and only if f3 < n — 2s. 
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Proof. Let r = \x\. It is easy to see that 

f' p (r) = -f3r(r 2 + l)"!" 1 , $(r) = f3(r 2 + l)^ 2 [(/3 + l)r 2 - l] . 
Hence, 

ffr) + n ~ 2S r +l f p {r) = /3(r 2 + Xy^ 2 [(/? - n + 2s)r 2 - (n - 2 a + 2)] . 

Suppose < /3 < n — 2s. Then the right-hand side of the preceding inequality is negative, and 
(II, 9j) holds. If < s < 1, this implies that fp is s-subharmonic. 

If j3 > n — 2s, then fg(r) + n ~ 2s+1 /a(r) is either positive or changes sign. Hence, (|1.9|) fails 
in this case. 

Moreover, by formula (|6.5p discussed below, 

<f>(r) = -(-A)V/9(r) = C(a, /3, n) F(a, 6, c, -r 2 ), 

where C(a, /3, n) is a positive constant, and 

i a n \ n + 2s L 2s + (3 n 

(4-2) « = ^-' 6 =^-' C =2' 

where i^(x) =2Fi(a,b,c,x) is the hyper geometric function. Notice that -F(O) = 1 and by a 
formula due to Gauss (see |AAR] . Theorem 2.2.2, p. 66): 

T(c — a)L{c — b) 
By Pfaff 's transformation, 

F(a, b, c, -r 2 ) = (1 + r 2 )- b F(c - a, b, c, -£— -). 

r z + 1 

It follows: 

(4.4) F(-r 2 )^(l + r 2 )" b l2JM 2 J asr^+oo. 

Hence, when n — 2s < f3 < n, < s < 1, (p(r) < for r large, and so changes sign. In other 
words, fp fails to be s-subharmonic in this case. □ 

5. Applications 

We first recall the following definitions. Let s G (0,1]. Let u be any continuous (or more 
generally upper semicontinuous) function u : M n — > EU {— oo} such that 

\u(x)\ 



" (1 + |x| 2 ) 2 



dx < +oo. 



We say that u is s-subharmonic in R n if 

-(-A) s u > in D'(R n ). 
Analogously we shall say that u is s-superharmonic in W 1 

-(-A) s u < in D'(R n ). 

Whenever u is both s-subharmonic and s-superharmonic in we shall say that u is s-harmonic in 
R n . 

We remark that from the representation given in Theorem 11.11 we can deduce a Liouville 
theorem for radial s-subharmonic functions. 
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Corollary 5.1. Assume that u is a positive continuous function, radially decreasing and s 6 
(0, 1]. Suppose that 

u(x) 

n + 2s 



f - 



-dx < +oo, 



(1+ | x I 2 )" 

and — (—A) s u > 0. // lim r _ i , +00 u(r) = then u = 0. 

Proof. We remark that if u is radially decreasing then u(r) > u(r£) and u{r) < tt(|) for every 
£ > 1. Hence, recalling Theorem 11.21 inequality 1 1.81 for every £ > 1 and for every positive r it 
results 

u{r^) — u{r) ~ 
On the other hand, for every fixed r, if £ — >■ +oo, then 

u(r)-u(?) 

Inn — -— ^- = +oo. 

£->+oo ii(f*£) — u(r) 

We recall that lim r _s. +00 u(r) = 0, so we get 

u(r) — u(0) 

hm w . , w = +oo, 
^->+oo —u(r) 

which implies a contradiction whenever u is bounded and not identically zero. □ 

The following result gives a derivative rule for fractional Laplacians. 
Theorem 5.2. Let s £ [0,1]. For every differentiable radial function u such that 

f l U ( X ) l 7 

/ 7 1 ,~, n+2s dx < +°°> 

JK. n (1+ | X \ z ) 2 

i/te following formula holds: 

re -i\ d(-AYu(r) 1 .... , 

(5.1) -i / — ^ = -(-A) s (-2su + n/). 

Proof. Differentiating (II. 7p . we obtain: 

d(-A)*u(r) 2s 

-, = - A ) u(r) 

dr r 

f+°° / u'( L ) \ 

+ r- l - 2s ^ f (u'(r) - u'irOZ + («'(r) - -^)r n+2s J £(£ 2 " l)" 1 " 2 ^ (« 

= _^£(_A) s n(r) 
r 



+ r- 2s f ™ I r- 1 ( (ru'(r) - u'(r£K) + (™'(r) - ^-) j £~ n+2s J £(£ 2 ~ l)- 1 ' 28 H(£)d£ 

= _^(_A) s n(r) + -(-A) s (ru'(r)) = -(-AY(-2su + ru'), 
r r r 

D 

If 

f k ( x ) = -(l + r 2 )-^- l \ 
then 

^(r) = 2(^-l)(l+r 2 )-^r 
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and recalling formula (|5.ip we get 

- 2sf k (r) + rf' k (r) = 2s(l + r 2 )'^^ + 2(|r - 1)(1 + r 2 )"t r 2 
=2(1 + r 2 )~5S ( s (l + r 2 ) + (|r ~ l)r 2 ) = 2(1 + r 2 )^ ^ + ( 5 + JL _ l) r : 
Thus 



dr 



-A) s (-2 S / fe + r/£) = -(-A) s ((1 + r 2 )"^ (s + (s + ^ - ljr 5 



= ; ("A) s ((i + - 2 )-- ((« + Y k ~ !) + ( s + Yk ~ l)r ' 

+ ^(-Ar((l + r 2 )-t( s - (s + ^_i : 

= 2(s+ ^~ 1} (-Ar (a + r 2 )-^- 1 )) - 2(t ~ 1)} (-Ar ((1 + ^ 2 )-^ 

In the next theorem we deduce the maximum principle for radial s-subharmonic functions. 

Theorem 5.3. Suppose that u is a radial s-subharmonic function s £ (0,1]. Then either u is 
constant or u can not realize a maximum in W 1 . 

Proof. Indeed, by contradiction, suppose that u realizes an absolute maximum in f G M n . Hence 
for every £ > 1, u(f) — u(f£) > 0, u(f) — u(?) > and as a consequence 

u(f) - u (f0 + (u{f) - u{-)\ £ 2s - n > 0. 

On the other hand 

-(-A) s (f) > 0, 
hence for every £ > 



u(f) - n(re) + 1 «(f) - n(-) ) e 2s_n = 

and in particular this implies u = u(f) because u(f) — u(f£) > and (u{f) — u(t)) > 0. D 

Corollary 5.4. If u is a radial s-harmonic function, then either u is constant or u does not 
realize either an absolute maximum or an absolute minimum. 

Corollary 5.5. Suppose u\ and U2 are two radial s-harmonic functions. If U\ > 112, then 
u\ > U2, or u\ = U2, i-e., u\ cannot intersect with U2I otherwise u\ coincides with U2- 

Proof. Indeed, if there exists f such that u\{f) = U2(f), then u\ — U2 has a minimum in f and 
ni — U2 is s-harmonic. Then u\ = U2 by the previous result. D 

Corollary 5.6. Suppose that u is a radial continuous s-harmonic function vanishing at infinity. 
Then u = 0. 



Proof. If sup K n u = then u realizes its minimum in W 1 . Then we can apply Corollary [57 
obtaining that u is constant and necessarely u = 0. Analogously if inf^n u = then u has to 
attain its maximum in M. n otherwise u = 0. Thus by Corollary 15.41 we conclude that u = 
indeed. D 

Corollary 5.7. Let U\,U2 be continuous radially symmetric functions such that 

(-A) s m = (-A) s u 2 . 

If lim r ^ +OQ (ui — 1*2) = then U\ = U2- 
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Proof. Recalling the linearity of the fractional Laplace operator we get: 

(-A) s (ui - « 2 ) = 0. 

Moreover lim r _ ) . +0O (^l — Uq) = 0. It follows by Corollary 15.71 that U\ = u^- D 

Our results make it possible to provide an answer to a question which was left open in 
[FFV| . More precisely, in [FFV| . Theorem 2.1, an inequality involving A:-convex functions and 
the fractional Laplace operator was proved. For reader's convenience we state below the result 
in [FFV] . 

Proposition 5.8 (Ferrari- Franchi- Verbitsky) . 1 < k < ?-, and let a = -^K. Suppose u G C 2 (IR n ) 
is a k— convex function on W 1 vanishing at oo. If 

(i) -(-A) a / 2 u > 0, 
(ii) (-A) a / 2 [-(-A) Q / 2 u] fc > 0, 
then there exists a positive constant C^^ n such that 

(5.2) / (-(-A) a/2 u) +l dx<C Kn I -uF k [u]dx. 



It was unclear if, for 1 < k < Q, the set 

J-Ck = {u G C (M ra ) : u, k — convex, vanishing at oo, (i) and (ii) hold}, 

was not trivial. Obviously is not empty because for any fc,l<fc<^,0G FCk- 

Theorems II. 21 and 1 1.31 in this paper will be employed to give a positive answer to this question. 
Indeed, 

f k (x) = -(1+ | x l 2 )-^- 1 ) 

is /c-convex, vanishing at oo, and — (— A) a ' 2 fk > 0, for every k G N such that ^ > k, i.e. fk 
satisfies (i). Moreover if k = 1, then a = 1, and condition (ii) becomes 

(_A) 1 /2[_ ( _ A) l/2 /l]=A/l > . 

The case k > 2, which is much harder, is considered in the next two sections. 

6. The iterated fractional Laplacian condition 
In this section we verify the iterated fractional Laplacian condition of Proposition 15. 8t 

(6.1) (-A)°/ 2 [-(-A) Q / 2 u] fc > 0, 

where a = -m-, 1 < k < ^, for the /c-convex function u(x) = —(1 + |j;| 2 )~^' 2 on M n with 
(3 = j — 2 > 0. As explained above, the case k = 1 is trivial, and from now on we will assume 
k > 2. We observe that u is an extremal function in the important Hessian Sobolev inequality of 
X.-J. Wang |W1| . |W2j . The Fourier transform of u is given by the radially symmetric function 

Kn=£(\Z\) 

(6.2) u£) = -C(p,n) * , £GIT, 

where C(/3, n) is a positive constant, and K^ is the modified Bessel function of order 7. 
It is possible to express (— A) a ' 2 u in terms of the Gaussian hyper geometric function 

F(a, b, c, x) = 2 Fi(a, b, c, x) 

discussed in the next section. For radially symmetric functions /(r), where r = |x|, the Fourier 
transform formula can be stated in the form: 

r+00 J n -2 (sr) 

(6.3) f(s) = (2^)"/ 2 / -^-/(r) r"- 1 dr, 

Jo (sr) 2 
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where s = |£|. Hence, from the preceding formulas we deduce: 

(6.4) -(-A) Q / 2 u(s) = C(a,/3,n)s^ / K„-i(r) J^i (sr) r 2+ a dr, 

Jo 2 2 

where C(a,(3,n) is a positive constant. 

Using the well-known integral involving the product of Bessel functions J 7 and modified Bessel 
functions K~, we obtain the explicit formula for (— A) a ' 2 u (see e.g., |MOS| . Sec. 3.8, p. 100): 

(6.5) -(-A) Q / 2 u(x) = C(a,P,n)F(a,b,c,-\x\ 2 ), x € K n , 
where C(a,f3,n) is a positive constant. The parameters a, b, and c are given by: 

n + a a + (3 n 

(6.6) a = ^— , 6=—^—, c =-, 

where 

2k n 

(6.7) a= fcTI' ^"fe" 2 ' n ~ 2k ' k ~ 2 - 

Notice that a > c > b > and a > 6 + 1 (a = 6 + 1 when k = 1) in our case. To verify ()6.ip . we 
need to demonstrate: 

(6.8) (-A)°/ 2 [F(a, b, c, -|x| 2 )] fc > 0. 
Let 

(6.9) <p(x) = F(a,b,c,-\x\ 2 ), x£R n . 

By (|4.4p , (j) clearly satisfies condition (|1.6p . Invoking Theorems 11.31 and 11.21 we see that 
follows from condition (|1.9p : 

([0(r)] fc ) // + n ~ a + 1 ([^(r)] fc ) / <O, r>0. 
r 

By direct differentiation, 

([0( r )] fc y + n ~ a + l {W)) k y = kwt 

r 
Hence it suffices to verify the inequality 



,»/ n /, -,\ Wi r )] 2 n — a + l ... . 

<f>"(r) + (k-l) ^±-f- + 41 {r) 

cp{r) r 



,i, x /, x [</>'(^)l 2 n — a + l ,, . 

6.10 0"(r) + (jfe - 1) L v, ; J + —<f> '(f) < 0, r > 0. 

0(r) r 

Since </»(r") = F(—r 2 ), where Fix) = F(a,b,c,x), we have <j)'(r) = —2rF'(—r 2 ) and <f>"(r) 
4r 2 F"(—r) — 2F'(—r 2 ). It follows that the preceding inequality is equivalent to: 

(6.11) F"(-r 2 ) + (k - 1) K[~ r T ~ n ~^ +2 F\-r 2 ) < 0, r > 0. 



F(-r 2 ) 2r 2 

-Q- 

2 



Letting x = r 2 , x > 0, and noticing that - ^ +2 = 2c — a + 1 by (|6.6p . we rewrite the preceding 
inequality as: 

(6.12) F"(-x) + (Ife - 1) [F i[~ X) } 2 - 2C ~ a + 1 F'(-x) < 0, x > 0. 

r (— x) x 

Using the hypergeometric equation ( [AAR| . p. 75): 

(6.13) x(l + x)F"(-x) - (c + (a + b + l)x) F'(-x) + abF(-x) = 0, 
we eliminate the second derivative in f)6. 12|) : 



(k-l)[F'(-x)] 2 + 



c+(a + b + l)x 2c-a + l 
x(l + x) x 



F'(-x)F(-x) - -^-[F(-x)] 2 < 0. 

X 1 + X 
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Letting A = pfegy ■, where x > 0, we reduce the preceding estimate to the quadratic inequality 

(6.14) x(l + x)(k- 1)A 2 + [(2a + b - 2c)x + a - c - 1] A - ab < 0, 
for x > 0. Prom the differentiation formula ( j A ARj . p. 94), 

(6.15) F'(a, b, c, x) = —F(a + 1, 6 + 1, c + 1, x), 

c 

it follows: 

teifil \ ab F(a + l,b + l,c+l,-x) ^ 

(6.16) A = — - — r >0, x >0. 

c b {a,b,c,—x) 

Solving inequality (|6.14[) for A > 0, we obtain: 



(6.17) < A < —r — -r— 7- s , x > 0. 

y ' ~ 2(k - l)x(l + x) 

where 

(6.18) d 1 = 2a + b-2c, d 2 = c-a + l, d 3 = 4ab(k - 1) = 4a(2c - a - b) 
by (|6.7p . Equivalently, 

2ab 

(6.19) < A < =, x > 0, 

dix - d 2 + y{d\x - d 2 ) 2 + d 3 x(l + x) 

Using (|6.16p . we see that (|6.14p is equivalent to: 



F(a,b,c,-x) dix - d 2 + y/{dix - d 2 ) 2 + d 3 x(l + x) 
(6 ' 20) F(a + l,6 + l,c+l,-x)- 2c ' * > °" 

We next use Pfaff's transformation ( |AAR| . Theorem 2.2.5, p. 68): 
(6.21) F{a,b,c,y) = (l-y)- i F(c-a,b,c,-^—), fory<l. 

Letting y = — x, x > 0, and applying (|6.2ip to both F(a, b, c, — x) and F(a + 1, b + 1, c + 1, — x) 
we get 

^■ ll) F(o + l,6 + l,c+l, -x) -^ + X >F(c-a,b + l,c+l,^ ri Y X> ^ 

Letting t = jxj, we rewrite (|6.20p in the equivalent form: 



23 F(c-a,b,c,t) > dit - da(l - t) + ygj ~ <k(l - t)) 2 + g < 

1 ' J F(c-a,6 + l,c+l,t) ~ 2c 

where di,d 2 ,d 3 are given by (|6.18p . We note that since (c — a) + b < c and (c — a) + (6 + 1) < c+1, 
it follows that the both hypergeometric functions in the ratio on the left-hand side are finite at 
t = l. By g3D, 



(6.24) F( C -a,b,c,l) = T ^-l F(c - a,b + l,c + 1, 1) = j& ± ]^ = 9 . 

r(a)r(c — o) r(a + l)r(c — o) 

Consequently, 

F(c — a, 6, c, 1) a 



(6.25) 



F(c-a,6 + l,c + l,l) c' 



In fact, since o > b + 1 for k > 2, both hypergeometric functions in the ratio are twice continu- 
ously differentiable for t < 1, including the endpoint £ = 1 when (see [AAR| . Theorem 2.3.2, p. 

78). 
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To prove (|6.23p . we express it in the form: 
(6.26) /(£) > g(t), < t < 1, 

where / is the Gaussian ratio: 

(6-27) f(t)= J7 / (C ~ a ' b : C ' t \ r , -oo<t<l, 

y ' Jy } F{c-a,b + l,c+l,ty 

and 



(6.28) g{t) = d l t- d2 jl-t) + ^ {dl t- d2{ l-t)f + d^ ^ Q 

The proof of (|6.26p relies on the important observation that / is convex (in fact, even loga- 
rithmically convex, as is proved in the next section) while g is concave: 



Lemma 6.1. The function g defined by J6.28)) is concave on [0, +oo 



Proof. Let h(t) = VAt 2 + Bt + C, then 



,.,,, 2At + B , ,,... 4AC-B 2 

h'{t) = , and h"{t) 



2^JAt 2 + Bt + & ' A(At 2 + Bt + C) 



Hence, h is concave if B 2 - AAC > 0. Let At 2 + Bt + C = (d\t - d 2 (l - t)) 2 + d 3 t, where 
d 1 ,d 2 ,d 3 are defined by (IBTTHD . Then clearly, A = {d x - d 2 ) 2 , B = d 3 - 2d 2 (di + d 2 ), C = d\, 
and At 2 + Bt + C > for t > since d 3 > 0. Clearly, 

£ 2 - 4^C = [d 3 - 2d 2 (d! + d 2 )] 2 - 4(4 + d 2 fd\ = d 3 [d 3 - Ad 2 {d x + tfc)], 

where 

d 3 -4d 2 (di + d 2 ) = a(2c-a-6)-(c-o + l)(o + 6-c+l) = (c+l)(c-6-l) > 0. 

Notice that c — 6 — 1 > 0. Indeed, from ()6.7[) it follows that n(A: — 1) > 2k since n > 2k and 
k > 2. Hence, 

n(fc - 1) - 2k + a n(k-l)-2k 
(6.29) c-t-1— S L >^_J >0. 

Thus, -B 2 — 4AC > 0, and consequently h is concave. Since g" = ^h" < 0, it follows that g is 
concave as well. □ 

Since / is convex by Theorem 17.11 below, and g is concave by Lemma 16. 11 it follows that the 
graph of / lies above the tangent line at t = 1, while the graph of g lies below the tangent line 
at 4 = 1. In other words, f(t) > /(l) + (t-l)/'(l), while g(t) < g{\) + (t- l)g'(l) for < t < 1. 
Recall that by (l6T25|) . /(I) = -. Notice that #(1) = - as well. Indeed, 



/^ d\ + y/d{ + d 3 
9(1) = Yc , 

where 

d\ + d 3 = a 2 - 2a(2c - a - b) + (2c - a -bf + 4a(2c - a - b) 

= a 2 + 2a(2c - a - b) + (2c - a - 6) 2 = (2c - 6) 2 . 
Since 2c > b by ()6.7p . we obtain: 



(6.30) Jd( + d 3 = 2c-b, 

and consequently, by f|6. 18j) . 

di + 2c - b 2a + b-2c + 2c-b a 
9{1) = 2c = 2c = c' 
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Thus, /(l) = 5(1), and to verify ()6.26p . it remains to show that /'(l) < </(l)- Let us deduce 
the formula: 

(6.31) /'(I) = 7^% 

c{a — b — 1) 

Invoking the differentiation formula (j6.15fl . we obtain: 

, . _ (c - a)b F(c - a + 1, b + 1, c + 1, t) (c - a)(b + 1) F(c - a + 1, b + 2, c + 2, i) F(c - a, 6, c, t) 
^ ~ c F(c-a,b + l,c + l,f) c + 1 F(c-a,b + l,c+l,i) 2 

Using (|6.25p . we deduce from the preceding formula: 

, _ (c - a)b F(c-a + l,b + l,c + l,l) _ a(c - a)(b + 1) F(c - a + 1, b + 2, c + 2, 1) 
' ^ ~ c F(c-o,6 + l,c + l,l) c(c+l) F(c - a, 6 + l,c+ 1,1) 

Applying Gauss's formula (|6.24|) for F(a,b,c, 1) ( [A AR] , Theorem 2.2.2, p. 66), we obtain 
, m = (c - a)b T(c + l)r(q - b - l)T(a + l)r(c - b) 

; u ~ c r(o)r(c-6)r(c + i)r(o-6) 

a(c - o)(p + 1) r(c + 2)r(o - fr - l)r(q + l)r(c - 6) 
c(c+l) r(q + l)r(c - 6)r(c + l)r(q - b) 

(c — a)ab a(c — a)(b + l) a(a — c) 

c(q — 6 — 1) c(a — b—1) c(a — b — 1) 

This proves (|6.3ip . 

On the other hand, by direct differentiation, 

(632> »M = j^'.+*+ V(dlt _ rf2(1 _ t)) 2 + (i3 J' 

Hence, 

(6.33) o (1) = — di + d 2 + —== . 

2c y 2 y/df + d 3 J 

We next show that, after simplification, we get: 

(ria\ 'm a ( c+1 ) 

(6 ' 34) 9 (1) = «2^b) ■ 

To prove this, recall that y/d'f + d% = 2c — b by f|6.30j) . and d\ = 2a + b — 2c = a — (2c — a — b), 
d\ + d 2 = a + b - c + 1, and d 3 = 4a(2c - a - b) by (J6T18J) . It follows: 

1 / ( 2a + 6-2c)(a + b-c + l) + 2fl(2c-fl-b) ^ 
*(!) = - (a + 6-c+l + ^ J. 

Simplifying further the right-hand side, we rewrite it as follows: 

(2c - b){a + b - c + 1) + (2a + b - 2c) (q + b - c + 1) + 2a(2c - a - b) 

2c(2c - b) 

2a(a + b-c + l) + 2a(2c - a - b) a(c+l) 



2c(2c - b) c(2c - b) ' 

This proves (|6.34p . 

Now it is not difficult to see that g'(l) > /'(l), i.e., 

(6.35) q(q-c) q(c + l) 



c(a-b-l) ~ c(2c-b)' 
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Indeed, by (|6.7p . 2c > b > and a — b — 1 = 2k > for k > 2. Hence (|6.35j) is equivalent to: 

(o - c)(2c - b) - (c+ l)(o - b - 1) < 0. 
By factoring, we rewrite this as: 

(a - c)(2c - 6) - (c + l)(a - 6 - 1) = ac - 2c 2 - ab + 26c + c - a + b + 1) 

= c(o - 2c) - b(a - 2c) - (a - 2c) - (c- 6 - 1) = (a- 2c - l)(c- 6 - 1) < 0. 

By ()6.29p . c — 6 — 1 > 0. On the other hand, since 1 < a < 2, we have: 

n + 2 — a 
a - 2c - 1 = < 0, 

This proves (|6.35|) . and consequently g'(l) > /'(l), which yields (|6.26p . 

These computations have been verified using Mathematical which was also used to check that 
inequality (|6.23|) holds for many concrete values of k and n. In the next section, we will complete 
the proof of this estimate by showing that /(£) is convex if t < 1. 

Remark 6.2. When k = 1 and a = 1, the function u(x) = —(—A) 2(1 + |x| 2 ) 2_n , n > 3, is 
2-subharmonic, i.e. (|6.ip holds, but inequality (jl.9p with s = | fails for r large. In other words, 
(II. 9h is sufficient but not necessary for a C 2 radial function to be s-subharmonic. 



Indeed, if k = 1, then a = 6+1, and consequently by (|6.24|) and (|6.31|) . it follows that 
/'(l) = +oo, where 

tM F(c-a,b,c,t) / + /i 

m = F(c-a,b + l,c + l,ty -°°<^ 1 ' 

Note that inequality (I6.12p . and hence (16.14[) . is equivalent to (16.23f) . i.e., f(t) > g(t) for t < 1, 

where g(t) is defined by <|Q?]> with d 3 = 0. Since /(l) = 5(1) = § , and 5 '(1) < +00 by (JS^U), 

this contradicts /'(l) = +00. 

7. Logarithmic convexity of the ratio of hypergeometric functions 

Let F(a,b,c,x) =2-^1(0, b, c,x) denote the hypergeometric function. We refer to |AAR| and 
MOSJ for the relevant theory. For c> b> 0, it can be defined by ([AAR], p. 65): 

r(6)r(c-6)y (i-sz) a 

in the complex plane with a cut from 1 to 00 along the positive real axis. We will show in 
this section that the Gaussian ratio ip(x) = p , fe^irj x \ is logarithmically convex, and hence 
convex, for x £ (—00, 1) if c > b > and — 1 < a < 0. Since a + 6 < c, it follows by (16.24[) that 
ip is continuous at x = 1 as well, and hence 99 is convex in (—00, 1], including the end-point. 
Applying this to f(x) = F(c-Tb+i C c+i x) wri ere a, b, c are given by (|6.6p so that c > b > 0, and 
c — a = —Tf £ (—1,0), we will conclude the proof of Theorem II. 4[ 

The proof of the following theorem employs a method developed recently in |KSlj (see also 
[KS2]) to prove monotonicity and log-concavity in parameters a,b,c of generalized hypergeo- 
metric functions and their ratios. 

Theorem 7.1. Let — 1 < a < 0, and c > b > 0. Then the function (p(x) = p , fe^rj x \ is 

logarithmically convex in (— 00, 1). 

Proof. Let x G (— 00, 1). Let f\{x) = F(a,b,c,x) and f2(x) = F(a, b + 1, c + l,x). Then 

0°6 ¥>)'(*) = W) - M and 



(lo g¥ 0"(x) 



,„ ,. _ /i(^) #(*) 



/i(x) / 2 (x) 





2 


r^(x)i 

[/ 2 (x)J 
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_ f'{{x)h{x) - f 2 '(x)h(x) [f{(x)f 2 (x) - ti(x)A(x)] • [f[(x)f 2 (x) + f 2 (x)A(x)} 
fi(x)f 2 (x) [h{x)f 2 {x)f 

Clearly, fi(x) > and f 2 (x) > by (J7TTJ) . Since a < and b > 0, c > 0, it follows by the 
differentiation formula (|6.15j) : 

f[(x) = — F(a + l,b + l,c+ l,x) <0, /£(s) = ° (6 + ^ F(o + 1, b + 2, c + 2, s) < 0, 
c c + 1 

and consequently, 

/I(s)/ 2 (x) + f 2 (x)fi(x) < 0, -oo < x < 1. 

Hence to show that (log(p)"(x) > 0, it suffices to prove the following inequalities: 

(7.2) fi(x)Mx)-f£(x)fi(x)>0, fi(x)f 2 (x)-f^x)f 1 (x)>0, -oo < x < 1. 

It will be more convenient to make a substitution £ = —x, and work with gi(t) = fi(—t), 
92(t) = fii—t) for * £ ( — 1) +00). Then the preceding inequalities are equivalent to: 

(7.3) gi(t)g 2 (t) - g' 2 (t)gi(t) < 0, g'i(t)g 2 (t) - g' 2 '{t) gi {t) > 0, -1< t < +00. 

We next employ the integral representation (|7.ip to express both g\ and g 2 in the form of 
Stieltjes-type transformations: 

gi (t)= s b ~ 1 (l + sty a w(s)ds, g 2 (t) = j s b (l + st)- a w(s)ds, -1 < t < +00, 
Jo b J 

where 

w(s)= ^ (c) (i- s y-*-\ o<s<i, 
r(6)r(c - 6) 

is a positive weight function. Then we have: 

^(t) = -a [ s b {l + sty 1 - 1 w(s) ds, g' 2 (t) = -a\ f s b+1 (1 + sty 1 - 1 w(s) ds, 
Jo ' b J 





a-2 , 



g»(t) = a(a + 1) I s b+1 (1 + sty 2 w{s) ds, g%(t) = a(a + 1) ~ I s b+2 (1 + sty-' 2 w(s) ds, 
Jo b j 

where — 1 < t < +00. 

To prove the first inequality in f|T.3[) . i.e., g' 1 (t)g 2 (t) < g' 2 (t)gi(t), we rewrite it in the equivalent 

form: 

-a\( f s b (1 + sty 1 " 1 w(s) ds] ( [ s b (1 + sty 1 w(s) ds 



/ wo 

1 \ / /■! 



< ~ CL l([ sb+1 ( l + st r a ~ lw ( s ) ds ) (I s h - 1 {l + st)- a w{s)ds\. 
Similarly, the second inequality in ([7.3j) . i.e., g'{{t)g 2 (t) > g 2 {t)g\{t), is equivalent to: 

a(a + l)~([ s b+1 (1 + sty- 2 w(s) ds) (I s b (1 + sty w{s) ds 



>a(a + l)j } i s b+2 (l + sty- 2 w(s)dsj (/ s 6 " 1 (1 + si)- a u>(s) ds 

Fix t € (— 1, +00). Since c > 6 > 0, and — 1 < a < 0, the first inequality above can be expressed 
in the form (see |KSlj . p. 341): 

(7.4) (J h(s)p(s)ds) (J q(s)p(s)ds) < (J q(s)h(s)p(s)ds) (J p(s)ds), 

where h(s) = j^, p(s) = s 6_1 (l + st)- a w(s), q(s) = s and w(s) = T ( b yr(l-b) i 1 ~ s Y~ b ~ l - 
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Analogously, the second inequality is equivalent to: 
(7.5) (j h 1 (s)p(s)ds] (j q(s)p(s)d S ] < (j q{s) h^s) p(s) ds) (j p{s)ds 

2 

where h\{s) = ; * .s» , p(s), q(s) and w(s) are as above. Since p(s) > 0, and the functions q(s), 
h(s), and hi(s) are increasing on (0, 1) for any fixed t > — 1, both (|7.4p and (|7.5p follow from 
Chebyshev's inequality for monotone functions (see |HLP] ). D 

Remark 7.2. The condition c > b > in Theorem 17.11 can be extended to c > min(a, 6) > 
using the symmetry of F{a, b, c, x) in a and b. 

Remark 7.3. Theorem 17.11 holds for the ratio of hyper geometric functions q+\F q ((a, b),c, x) in 
place of 2-F1 (a, b, c, x), where — 1 < a < and C& > 6^ > 0, k = 1,2, . . . ,q, q > 2. 

The proof of the logarithmic convexity of the ratio of q +\F q is similar to the proof of Theo- 
rem [7j] using the corresponding Stieltjes type integral representation ( [KS1] , Lemma 1). 

Corollary 7.4. Let < a — c < 1, and c > b > 0. T/ien the ratio „, , 1 ^ , 'f' 1 , = — r is convex in 

J F(a+l,b+l,c+l,x) 

(-oo,l). 

Proof. Let Vl (-x) = F{a +i^^. ltX) , x < 1. By @22J, 

X 

(/?l(x) = (1 + X)V9 ( — "), X>~1, 

X + 1 

where 

F(c-a,b,c,y) 

F(c- a, 6 + l,c+ l,y) 

We deduce: 

where ipo is logarithmically convex by Theorem 17. 1| and consequently convex, so that v?o(y) — ^j 
y < 1. Then, obviously, yi(— x) is convex as well. D 
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